In this work, we explore the potential of large language models (LLMs) for generating functional test scripts, which necessitates understanding the dynamically evolving code structure of the target software. To achieve this, we propose a case-based reasoning (CBR) system utilizing a 4R cycle (i.e., retrieve, reuse, revise, and retain), which maintains and leverages a case bank of test intent descriptions and corresponding test scripts to facilitate LLMs for test script generation. To improve user experience further, we introduce Re4, an optimization method for the CBR system, comprising reranking-based retrieval finetuning and reinforced reuse finetuning. Specifically, we first identify positive examples with high semantic and script similarity, providing reliable pseudo-labels for finetuning the retriever model without costly labeling. Then, we apply supervised finetuning, followed by a reinforcement learning finetuning stage, to align LLMs with our production scenarios, ensuring the faithful reuse of retrieved cases. Extensive experimental results on two product development units from Huawei Datacom demonstrate the superiority of the proposed CBR+Re4. Notably, we also show that the proposed Re4 method can help alleviate the repetitive generation issues with LLMs.

With the release of ML.NET, a API that C# developers can use to infuse their applications with machine learning capability, I've been keen to combine my knowledge of Azure Functions with the API to build some wacky serverless machine learning applications that would allow me to enhance my GitHub profile and cater to all the buzzword enthusiasts out there! This post won't be a tutorial. I'm writing this more as a retrospective of the design decisions I took while building the application and the things I learnt about how different components work. Should you read this and decide to build upon it for your real world applications, hopefully you can apply what I've learnt in your projects or better yet, expand on the ideas and scenarios I was working with. I'll be focusing more on what I learnt about the ML.NET API itself rather than spending too much time about how Azure Functions work.

Machine Learning (ML) powers an increasing number of the applications and services that we use daily. For organizations who are beginning to leverage datasets to generate business insights -- the next step after you've developed and trained your model is deploying the model to use in a production scenario. That could mean integration directly within an application or website, or it may mean making the model available as a service. As ML continues to mature the emphasis starts to shift from development towards deployment. You need to transition from developing models to real world production scenarios that are concerned with issues of inference performance, scaling, load balancing, training time, reproducibility and visibility.

Machine learning (ML) powers an increasing number of the applications and services that we use daily. For organizations who are beginning to leverage datasets to generate business insights, the next step after you've developed and trained your model is deploying the model to use in a production scenario. That could mean integration directly within an application or website, or it may mean making the model available as a service. As ML continues to mature, the emphasis starts to shift from development towards deployment, you need to transition from developing models to real-world production scenarios that are concerned with issues of inference performance, scaling, load balancing, training time, reproducibility, and visibility. In previous posts, we've explored the ability to save and load trained models with TensorFlow that allow them to be served for inference.

In our webinar "Optimizing Machine Learning with TensorFlow" we gave an overview of some of the impressive optimizations Intel has made to TensorFlow when using their hardware. You can find a link to the archived video here. During the webinar, Mohammad Ashraf Bhuiyan, Senior Software Engineer in Intel's Artificial Intelligence Group, and myself spoke about some of the common use cases that require optimization as well as benchmarks demonstrating order-of-magnitude speed improvements when running on Intel hardware. TensorFlow, Google's library for machine learning (ML), has become the most popular machine learning library in a fast-growing ecosystem. This library has over 77k stars on GitHub and is widely used in a growing number of business critical applications.

XGBoost is a library designed and optimized for tree boosting. Gradient boosting trees model is originally proposed by Friedman et al. By embracing multi-threads and introducing regularization, XGBoost delivers higher computational power and more accurate prediction. More than half of the winning solutions in machine learning challenges hosted at Kaggle adopt XGBoost (Incomplete list). XGBoost has provided native interfaces for C, R, python, Julia and Java users.